<html>
<head><meta charset="utf-8"><title>ABI not &quot;normalized&quot;? · miri · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/index.html">miri</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html">ABI not &quot;normalized&quot;?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="242698036"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242698036" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> hyd-dev <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242698036">(Jun 15 2021 at 07:42)</a>:</h4>
<p>When I was trying <code>intrinsics::r#try</code> for <a href="#narrow/stream/269128-miri/topic/.60panic.3Dunwind.60.20crates.20can.20depend.20on.20.60panic.3Dabort.60.20crates/near/242695194">https://rust-lang.zulipchat.com/#narrow/stream/269128-miri/topic/.60panic.3Dunwind.60.20crates.20can.20depend.20on.20.60panic.3Dabort.60.20crates/near/242695194</a>, I noticed that Miri was not happy if I use closures:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#![feature(core_intrinsics)]</span><span class="w"></span>

<span class="k">use</span><span class="w"> </span><span class="n">std</span>::<span class="p">{</span><span class="n">intrinsics</span><span class="p">,</span><span class="w"> </span><span class="n">ptr</span><span class="p">};</span><span class="w"></span>

<span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">unsafe</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">        </span><span class="n">intrinsics</span>::<span class="n">r</span>#<span class="kr">try</span><span class="p">(</span><span class="o">|</span><span class="n">_</span><span class="o">|</span><span class="w"> </span><span class="fm">todo!</span><span class="p">(),</span><span class="w"> </span><span class="n">ptr</span>::<span class="n">null_mut</span><span class="p">(),</span><span class="w"> </span><span class="o">|</span><span class="n">_</span><span class="p">,</span><span class="w"> </span><span class="n">_</span><span class="o">|</span><span class="w"> </span><span class="fm">todo!</span><span class="p">());</span><span class="w"></span>
<span class="w">    </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>
<div class="codehilite"><pre><span></span><code>error: Undefined Behavior: calling a function with ABI rust-call using caller ABI Rust
 --&gt; src/main.rs:7:9
  |
7 |         intrinsics::r#try(|_| todo!(), ptr::null_mut(), |_, _| todo!());
  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ calling a function with ABI rust-call using caller ABI Rust
  |
  = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
  = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information

  = note: inside `main` at src/main.rs:7:9
  = note: inside `&lt;fn() as std::ops::FnOnce&lt;()&gt;&gt;::call_once - shim(fn())` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::&lt;fn(), ()&gt;` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:125:18
  = note: inside closure at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:49:18
  = note: inside `std::ops::function::impls::&lt;impl std::ops::FnOnce&lt;()&gt; for &amp;dyn std::ops::Fn() -&gt; i32 + std::marker::Sync + std::panic::RefUnwindSafe&gt;::call_once` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:259:13
  = note: inside `std::panicking::r#try::do_call::&lt;&amp;dyn std::ops::Fn() -&gt; i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32&gt;` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:401:40
  = note: inside `std::panicking::r#try::&lt;i32, &amp;dyn std::ops::Fn() -&gt; i32 + std::marker::Sync + std::panic::RefUnwindSafe&gt;` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:365:19
  = note: inside `std::panic::catch_unwind::&lt;&amp;dyn std::ops::Fn() -&gt; i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32&gt;` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:434:14
  = note: inside `std::rt::lang_start_internal` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:34:21
  = note: inside `std::rt::lang_start::&lt;()&gt;` at /playground/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:48:5
</code></pre></div>
<p>I guess that's because Miri <a href="https://github.com/rust-lang/miri/blob/d39f0c64b8b369188a73a655716ab56683a6537b/src/helpers.rs#L168">does not</a> <a href="https://github.com/rust-lang/rust/blob/d74b36ea2f814b720c39d7b60aecaefe512a056b/compiler/rustc_mir/src/interpret/terminator.rs#L268">normalize the ABI</a>, but:</p>
<ul>
<li>Is it really fine to use a closure here (so it's right for Miri to normalize the ABI)?</li>
<li>If it is, the <code>std::panic::catch_unwind</code> implementation does not use closures (and other shim callers also do not seem to use them), so it does not seem necessary to normalize the ABI (at least for now), but should Miri still do it to be consistent or to make testing and experimenting more convenient?</li>
</ul>



<a name="242698818"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242698818" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242698818">(Jun 15 2021 at 07:51)</a>:</h4>
<blockquote>
<p>Is it really fine to use a closure here (so it's right for Miri to normalize the ABI)?</p>
</blockquote>
<p>The comment says the ABIs are all the same, so I think this is fine, but I am not sure. ABIs are mostly a mystery to me.^^</p>



<a name="242698879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242698879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242698879">(Jun 15 2021 at 07:51)</a>:</h4>
<p>I also noticed that <a href="https://github.com/rust-lang/rust/blob/d74b36ea2f814b720c39d7b60aecaefe512a056b/compiler/rustc_mir/src/interpret/terminator.rs#L357">here</a> we have a slightly different list of ABIs than above where we normalize. That doesn't seem great.</p>



<a name="242698949"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242698949" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242698949">(Jun 15 2021 at 07:52)</a>:</h4>
<p>We should probably factor normalization into a separate function that can then also be called by Miri.</p>



<a name="242698986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242698986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242698986">(Jun 15 2021 at 07:52)</a>:</h4>
<p>Cc <span class="user-mention" data-user-id="119009">@eddyb</span> ; IIRC you told me that RustCall and Rust are the same ABI so it is fine to mix-and-match them</p>



<a name="242699019"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699019" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699019">(Jun 15 2021 at 07:53)</a>:</h4>
<p>yeah <code>RustCall</code> is not even an ABI, it's... really silly. god I need to finish fixing that whole mess</p>



<a name="242699062"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699062" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699062">(Jun 15 2021 at 07:53)</a>:</h4>
<p>ideally monomorphization would just remove any trace of a function <del>call</del> signature being variadic, type-wise, and only leave an indication in the <code>TerminatorKind::Call</code> that a tuple needs to be expanded into its components</p>



<a name="242699155"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699155" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699155">(Jun 15 2021 at 07:54)</a>:</h4>
<p>the "ABI" is being abused to indicate like 3 different things and it's really unhelpful</p>



<a name="242699668"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699668" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699668">(Jun 15 2021 at 08:00)</a>:</h4>
<ol>
<li><code>extern "rust-call" fn(X, Y, Args)</code> is basically <code>fn(X, Y, ...Args)</code> which Rust doesn't currently have, but also it doesn't transform whenever <code>Args</code> is known as a tuple, which means it has to be manually handled in so many places</li>
<li><code>extern "rust-call" fn</code> <em>definitions</em> auto-tuple the "variadic" part of the arguments, as if there was a <code>...args: Args</code> pattern (again, not in Rust yet) in the function signature</li>
<li><em>calls</em> to <code>extern "rust-call"</code> functions/<code>fn</code> pointers have to take the tuple that accounts for the "variadic" part and "spread it out", so that the actual ABI only sees its components as arguments</li>
</ol>



<a name="242699724"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699724" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699724">(Jun 15 2021 at 08:01)</a>:</h4>
<p>only 2. actually has an explicit non-ABI representation in <code>rustc</code>, i.e. the <code>spread_arg</code> field of <code>mir::Body</code>: <a href="https://github.com/rust-lang/rust/blob/9089771daf6b1f1824446cca3306d7c18084eae0/compiler/rustc_middle/src/mir/mod.rs#L206-L210">https://github.com/rust-lang/rust/blob/9089771daf6b1f1824446cca3306d7c18084eae0/compiler/rustc_middle/src/mir/mod.rs#L206-L210</a></p>



<a name="242699786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242699786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242699786">(Jun 15 2021 at 08:01)</a>:</h4>
<p>tho it should probably be called <code>unspread</code> or <code>gather</code>/<code>collect</code> instead of <code>spread</code></p>



<a name="242709173"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709173" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709173">(Jun 15 2021 at 09:37)</a>:</h4>
<p><span class="user-mention" data-user-id="119009">@eddyb</span> so what does this all mean? calling <code>try</code> with a closure (coerced to a fn ptr since it doesnt capture anything) is fine without any special handling on the side of <code>try</code>, or not?</p>



<a name="242709355"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709355" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709355">(Jun 15 2021 at 09:39)</a>:</h4>
<p>all you have to do is expand out the type-level <code>FnSig</code> arguments (replacing the final tuple with its components) and replace <code>RustCall</code> with <code>Rust</code></p>



<a name="242709383"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709383" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709383">(Jun 15 2021 at 09:39)</a>:</h4>
<p>(things the typesystem should be doing for you automatically)</p>



<a name="242709465"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709465" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709465">(Jun 15 2021 at 09:40)</a>:</h4>
<p>and so the actual argument passing should happen on e.g. those tuple components, as individual arguments, and never the tupled up form</p>



<a name="242709581"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709581" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709581">(Jun 15 2021 at 09:41)</a>:</h4>
<p>the captureless closure coercion thing has one more caveat which is that it <em>also</em> abuses the fact that ZST arguments aren't ever passed to account for the <code>self</code> argument of <code>FnOnce::call_once</code>, but I'm guessing you handle that already?</p>



<a name="242709831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242709831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242709831">(Jun 15 2021 at 09:43)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> as an example, the signature of <code>&lt;Closure as FnOnce&lt;(A, B, C)&gt;&gt;::call_once</code> is:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">extern</span><span class="w"> </span><span class="s">"rust-call"</span><span class="w"> </span><span class="k">fn</span><span class="p">(</span><span class="n">Closure</span><span class="p">,</span><span class="w"> </span><span class="p">(</span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">C</span><span class="p">))</span><span class="w"></span>
</code></pre></div>
<p>and what I'm saying is that for everything to work out you "just" need to make sure you pretend it is:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">fn</span><span class="p">(</span><span class="n">Closure</span><span class="p">,</span><span class="w"> </span><span class="n">A</span><span class="p">,</span><span class="w"> </span><span class="n">B</span><span class="p">,</span><span class="w"> </span><span class="n">C</span><span class="p">)</span><span class="w"></span>
</code></pre></div>



<a name="242710112"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242710112" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242710112">(Jun 15 2021 at 09:46)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> going further, taking those as <code>fn</code> pointer types, transmuting between the two should always work and any calls should also behave indistinguishably. they are, after all, <em>the same type</em> in all but name</p>



<a name="242710177"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242710177" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242710177">(Jun 15 2021 at 09:47)</a>:</h4>
<p>a couple months ago I got roped into starting to fix this stuff but then got too busy with other things again :(</p>



<a name="242710225"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242710225" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242710225">(Jun 15 2021 at 09:47)</a>:</h4>
<p>(the correct internal representation is a "variadic (tuple) generics"-oriented one, regardless of how we'll represent that in the surface language)</p>



<a name="242710929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242710929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242710929">(Jun 15 2021 at 09:53)</a>:</h4>
<p><span class="user-mention silent" data-user-id="119009">eddyb</span> <a href="#narrow/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F/near/242709465">said</a>:</p>
<blockquote>
<p>and so the actual argument passing should happen on e.g. those tuple components, as individual arguments, and never the tupled up form</p>
</blockquote>
<p>yeah our function call / argument passing code does that</p>



<a name="242711070"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/269128-miri/topic/ABI%20not%20%22normalized%22%3F/near/242711070" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/269128-miri/topic/ABI.20not.20.22normalized.22.3F.html#242711070">(Jun 15 2021 at 09:54)</a>:</h4>
<p>that <em>should</em> be enough in general, tho small details could still be missing in edge cases, ofc</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>